refactor: simplify components.json code #7742

chewi · 2026-01-27T16:00:13Z

What this PR does / why we need it:

Firstly, this drops compatibility fallbacks for versions and multiArchVersions in components.json. There are no more instances of these.

Secondly, this massively simplifies the associated code involving jq. It's a powerful tool. 🙂

Lastly, this allow the OS variant to be used in components.json for OS Guard.

testPackagesInstalled() was not testing OS Guard properly because OS and OS_VERSION were being set to Ubuntu and 3.0, matching nothing from components.json.

Several components are not installed for OS Guard, but rather than add yet more shell code to handle this, simplify things by making it possible to use the OS variant in components.json. OS Guard's VARIANT_ID is osguard, but this is upper-cased by our scripts. Use DEFAULT to match an OS without VARIANT_ID.

There are no more instances of this. Signed-off-by: James Le Cuirot <[email protected]>

jq is very powerful, you know. :) Signed-off-by: James Le Cuirot <[email protected]>

We almost always pass `${OS}` so make this the default.

testPackagesInstalled() was not testing OS Guard properly because OS and OS_VERSION were being set to Ubuntu and 3.0, matching nothing from components.json. Several components are not installed for OS Guard, but rather than add yet more shell code to handle this, simplify things by making it possible to use the OS variant in components.json. OS Guard's VARIANT_ID is osguard, but this is upper-cased by our scripts. Use DEFAULT to match an OS without VARIANT_ID. Signed-off-by: James Le Cuirot <[email protected]>

Signed-off-by: James Le Cuirot <[email protected]>

Copilot

Pull request overview

This PR refactors the components.json handling to simplify the code and add support for OS variants (specifically OS Guard). The changes eliminate fallback compatibility code for deprecated versions and multiArchVersions fields, streamline jq-based JSON parsing, and enable OS variant-specific package configurations.

Changes:

Removed fallback support for deprecated versions and multiArchVersions fields in components.json
Simplified package version and URL retrieval using more efficient jq filters
Added OS variant support to distinguish between standard Azure Linux and Azure Linux OS Guard configurations
Updated components.json to use azurelinux/DEFAULT and azurelinux/OSGUARD variants for GPU-related packages
Removed OS Guard-specific conditional logic from installation scripts by using <SKIP> markers in components.json

Reviewed changes

Copilot reviewed 24 out of 80 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
parts/linux/cloud-init/artifacts/cse_helpers.sh	Core refactor: replaced `updateRelease()` with `getPackageJSON()`, simplified version/URL extraction with jq
parts/linux/cloud-init/artifacts/cse_install.sh	Added OS_VARIANT parameter to package lookup functions
parts/linux/cloud-init/artifacts/ubuntu/cse_install_ubuntu.sh	Pass OS_VARIANT to credential provider installation
parts/linux/cloud-init/artifacts/mariner/cse_install_mariner.sh	Pass OS_VARIANT to credential provider installation
parts/linux/cloud-init/artifacts/azlosguard/cse_install_osguard.sh	Pass OS_VARIANT to credential provider installation
vhdbuilder/packer/install-dependencies.sh	Removed OS Guard conditional checks for containerd, oras, and GPU packages
vhdbuilder/packer/test/linux-vhd-content-test.sh	Added OS Guard SKU detection and OS_VARIANT initialization
parts/common/components.json	Added OSGUARD variant with SKIP markers, changed 3 azurelinux entries to azurelinux/DEFAULT
schemas/components.cue	Added azurelinux/DEFAULT and azurelinux/OSGUARD to schema
pkg/vhdbuilder/datamodel/component_configs.go	Changed MultiArchVersions from string array to struct array
pkg/vhdbuilder/datamodel/component_configs_test.go	Updated test expectations for new multiArchVersionsV2 structure
e2e/components/components.go	Updated to check for DEFAULT variant when reading package versions
spec/parts/linux/cloud-init/artifacts/test_components.json	Removed deprecated multiArchVersions and versions test data
spec/parts/linux/cloud-init/artifacts/cse_helpers_spec.sh	Updated tests to use new getPackageJSON function, fixed typo
parts/linux/cloud-init/artifacts/README-COMPONENTS.md	Added documentation about OS variant support

Copilot · 2026-01-28T18:41:58Z

parts/common/components.json

          }
        },
-        "azurelinux": {
+        "azurelinux/DEFAULT": {


The PR changes only 3 instances of "azurelinux" to "azurelinux/DEFAULT" (at lines 1708, 1742, and 1776), but leaves 7 other instances unchanged. This inconsistency means that most azurelinux configurations are not explicitly marked as DEFAULT variant. This could lead to confusion about which packages support the OSGUARD variant and which don't. Consider either: (1) updating all azurelinux keys to azurelinux/DEFAULT for consistency, or (2) documenting why only these three packages (datacenter-gpu-manager-4-core, datacenter-gpu-manager-4-proprietary, dcgm-exporter) need the DEFAULT distinction.

Suggested change

"azurelinux/DEFAULT": {

"azurelinux": {

Copilot · 2026-01-28T18:41:58Z

schemas/components.cue

+	"azurelinux/DEFAULT"?: #AzureLinuxOSDistro
+	"azurelinux/OSGUARD"?: #AzureLinuxOSDistro
+	azurelinux?:           #AzureLinuxOSDistro


The schema defines both "azurelinux/DEFAULT" and azurelinux as valid keys in the same schema. This creates ambiguity and potential conflicts. Lines 90 and 92 define overlapping keys where azurelinux could match both patterns. Consider removing line 92 to avoid ambiguity since the code now explicitly handles the DEFAULT variant.

Copilot · 2026-01-28T18:41:58Z

spec/parts/linux/cloud-init/artifacts/cse_helpers_spec.sh

-            containerImage=$(readContainerImage "mcr.microsoft.com/dummyImageWithOldMultiArchVersions")
-            When call updateMultiArchVersions "$containerImage"
-            The variable MULTI_ARCH_VERSIONS[@] should equal "dummyVersion3 dummyVersion4 dummyVersion5"
+            The variable MULTI_ARCH_VERSIONS[@] should equal "dummyVersion1.1 dummyVersion1 dummyVersion2.1 dummyVersion2"


The test on line 137 expects multi-arch versions in a different order than before ("dummyVersion1.1 dummyVersion1 dummyVersion2.1 dummyVersion2" vs the old "dummyVersion1.1 dummyVersion2.1 dummyVersion1 dummyVersion2"). This suggests the new jq logic using map(.latestVersion, .previousLatestVersion) changes the ordering. While this may not be functionally critical, it's worth documenting whether the order matters for any consumers of this data.

Copilot · 2026-01-28T18:41:59Z

pkg/vhdbuilder/datamodel/component_configs_test.go

+			Expect(imageList).To(HaveLen(3))
 			Expect(imageList[0]).To(Equal("mcr.microsoft.com/unittest/mockimage:mock.000000.1"))
 			Expect(imageList[1]).To(Equal("mcr.microsoft.com/unittest/mockimage:mock.000000.2"))
+			Expect(imageList[2]).To(Equal("mcr.microsoft.com/unittest/mockimage:mock.000000.1"))


The test expects 3 images in the image list (line 223) but the comment on line 226 suggests this is the "previousLatestVersion" from mock.000000.1. However, looking at the mock data, mock.000000.1 is used as both an amd64OnlyVersion and as a previousLatestVersion in the multiArchVersionsV2. This creates duplicate entries in the image list. While this matches the implementation in ToImageList(), consider whether this duplication is intentional or if the test data should be restructured to avoid confusion.

Copilot · 2026-01-28T18:41:59Z

vhdbuilder/packer/test/linux-vhd-content-test.sh

+    elif [ "$OS_SKU" = "AzureLinuxOSGuard" ]; then
+      OS=$AZURELINUX_OS_NAME
+      OS_VARIANT=OSGUARD
    else
      OS=$UBUNTU_OS_NAME
    fi
    PACKAGE_VERSIONS=()
-    updatePackageVersions "${p}" "${OS}" "${OS_VERSION}"
+    updatePackageVersions "${p}" "${OS}" "${OS_VERSION}" "${OS_VARIANT}"
    PACKAGE_DOWNLOAD_URL=""
-    updatePackageDownloadURL "${p}" "${OS}" "${OS_VERSION}"
+    updatePackageDownloadURL "${p}" "${OS}" "${OS_VERSION}" "${OS_VARIANT}"


The variable OS_VARIANT is set only for the AzureLinuxOSGuard case (line 209) but is used unconditionally in function calls on lines 214 and 216. For other OS types (Ubuntu, Mariner, standard AzureLinux), OS_VARIANT will be undefined, which could cause unexpected behavior in the shell functions. Consider initializing OS_VARIANT="" before the OS_SKU conditionals to ensure it always has a defined value.

Copilot · 2026-01-28T18:41:59Z

parts/linux/cloud-init/artifacts/cse_helpers.sh

+    # By default, use the first match from these:
+    # .downloadURIs.${osLowerCase}/${osVariant}.current
+    # .downloadURIs.${osLowerCase}.current
+    # .downloadURIs.default.current
+    local search=".downloadURIs.\"${osLowerCase}/${osVariant}\".current // .downloadURIs.${osLowerCase}.current // .downloadURIs.default.current"
+
+    # For AZURELINUX, check the OS version (e.g. 3.0) prefixed with "v" before "current" (e.g. v3.0).
    if isMarinerOrAzureLinux "${os}"; then
-        if [ "$(echo "${package}" | jq -r ".downloadURIs.${osLowerCase}.\"v${osVersion}\"" 2>/dev/null)" != "null" ]; then
-            RELEASE="\"v${osVersion}\""
-        fi
-        return 0
-    fi
-    local osVersionWithoutDot=$(echo "${osVersion}" | sed 's/\.//g')
-    if [ "$(echo "${package}" | jq -r ".downloadURIs.ubuntu.r${osVersionWithoutDot}" 2>/dev/null)" != "null" ]; then
-        RELEASE="\"r${osVersionWithoutDot}\""
+        search=".downloadURIs.\"${osLowerCase}/${osVariant}\".\"v${osVersion}\" // .downloadURIs.${osLowerCase}.\"v${osVersion}\" // ${search}"
+    # For UBUNTU, check the OS version (e.g. 20.04) with no dots and prefixed with "r" before "current" (e.g. r2004).
+    elif isUbuntu "${os}"; then
+        search=".downloadURIs.\"${osLowerCase}/${osVariant}\".\"r${osVersion//.}\" // .downloadURIs.${osLowerCase}.\"r${osVersion//.}\" // ${search}"
    fi


The getPackageJSON function uses compound jq filter with the // operator for fallback logic. While this is elegant, the complex chained filter on line 868 is difficult to read and maintain. Consider adding inline comments explaining the precedence order, especially since this is a critical function that determines which package configuration is used.

Copilot · 2026-01-28T18:41:59Z

parts/linux/cloud-init/artifacts/cse_helpers.sh

+    # .downloadURIs.${osLowerCase}/${osVariant}.current
+    # .downloadURIs.${osLowerCase}.current
+    # .downloadURIs.default.current
+    local search=".downloadURIs.\"${osLowerCase}/${osVariant}\".current // .downloadURIs.${osLowerCase}.current // .downloadURIs.default.current"


The parameter expansion ${os_variant} is used in the search string without quotes. If os_variant is empty or contains spaces, this could lead to unexpected jq filter construction. The filter should handle empty variant values gracefully. Consider documenting the expected behavior when osVariant is empty.

Copilot · 2026-01-28T18:42:00Z

parts/linux/cloud-init/artifacts/cse_helpers.sh

    local componentName="$2"
    local os="$3"
    local os_version="$4"
+    local os_variant=${5:-DEFAULT}


The new getLatestPkgVersionFromK8sVersion function parameter os_variant defaults to "DEFAULT" on line 901, but this is inconsistent with other functions where empty string is used. For example, in getPackageJSON, an empty osVariant is expected. This inconsistency could cause issues with the fallback logic in getPackageJSON which checks for ${osLowerCase}/${osVariant} first. Recommend using an empty string as the default to maintain consistency.

Suggested change

local os_variant=${5:-DEFAULT}

local os_variant="${5:-}"

Copilot · 2026-01-28T18:42:00Z

pkg/vhdbuilder/datamodel/component_configs_test.go

 			Expect(components.ContainerImages[1].DownloadURL).To(Equal("mcr.microsoft.com/unittest/mockimage2:*"))
 			Expect(components.ContainerImages[1].Amd64OnlyVersions).To(HaveLen(0))
 			Expect(components.ContainerImages[1].MultiArchVersions).To(HaveLen(1))
-			Expect(components.ContainerImages[1].MultiArchVersions[0]).To(Equal("mock.000000.2"))
+			Expect(components.ContainerImages[1].MultiArchVersions[0].LatestVersion).To(Equal("mock.000000.2"))
+			Expect(components.ContainerImages[0].MultiArchVersions[0].PreviousLatestVersion).To(BeEmpty())


The test expectation on line 205 checks MultiArchVersions[0].PreviousLatestVersion but the index [0] refers to the first container image, not the first element being asserted. This appears to be a copy-paste error - it should check MultiArchVersions[1].PreviousLatestVersion to match the pattern from line 204 which correctly checks ContainerImages[1].MultiArchVersions[0].

Copilot · 2026-01-28T18:42:00Z

e2e/components/components.go

+		// Check if versionsV2 exists. Assume the DEFAULT OS variant.
+		versions := packageItem.Get(fmt.Sprintf("%s/DEFAULT.%s.versionsV2", distro, release)).Array()
+		if len(versions) == 0 {
+			versions = packageItem.Get(fmt.Sprintf("%s.%s.versionsV2", distro, release)).Array()
+		}


The e2e test code assumes DEFAULT variant when checking package versions (line 47), but there's no fallback to handle cases where neither distro/DEFAULT nor distro paths exist. If a package is defined with only a specific variant (e.g., only azurelinux/OSGUARD), this function would return an empty array for the DEFAULT variant. Consider adding error handling or documentation about expected behavior when the DEFAULT variant is not defined.

Suggested change

// Check if versionsV2 exists. Assume the DEFAULT OS variant.

versions := packageItem.Get(fmt.Sprintf("%s/DEFAULT.%s.versionsV2", distro, release)).Array()

if len(versions) == 0 {

versions = packageItem.Get(fmt.Sprintf("%s.%s.versionsV2", distro, release)).Array()

}

// Check if versionsV2 exists. Prefer the DEFAULT OS variant, then non-variant, then any variant.

versions := packageItem.Get(fmt.Sprintf("%s/DEFAULT.%s.versionsV2", distro, release)).Array()

if len(versions) == 0 {

versions = packageItem.Get(fmt.Sprintf("%s.%s.versionsV2", distro, release)).Array()

}

if len(versions) == 0 {

// Fallback: handle cases where only specific OS variants (e.g., azurelinux/OSGUARD) are defined.

versions = packageItem.Get(fmt.Sprintf("%s/*.%s.versionsV2", distro, release)).Array()

}

chewi temporarily deployed to test January 27, 2026 16:00 — with GitHub Actions Inactive

github-actions bot added the components This pull request updates cached components on Linux or Windows VHDs label Jan 27, 2026

chewi temporarily deployed to test January 27, 2026 16:03 — with GitHub Actions Inactive

chewi force-pushed the chewi/simplify-components branch from b43d562 to 33c1e10 Compare January 27, 2026 16:14

chewi temporarily deployed to test January 27, 2026 16:14 — with GitHub Actions Inactive

chewi force-pushed the chewi/simplify-components branch from 33c1e10 to 2422aa9 Compare January 27, 2026 17:03

chewi temporarily deployed to test January 27, 2026 17:03 — with GitHub Actions Inactive

chewi added 2 commits January 28, 2026 11:45

refactor: drop compatibility fallback for "versions" in components.json

d45dcb0

There are no more instances of this. Signed-off-by: James Le Cuirot <[email protected]>

refactor: drop fallback for "multiArchVersionsV2" in components.json

5172764

There are no more instances of this. Signed-off-by: James Le Cuirot <[email protected]>

chewi force-pushed the chewi/simplify-components branch from 2422aa9 to 09b286b Compare January 28, 2026 18:36

Copilot AI review requested due to automatic review settings January 28, 2026 18:36

chewi temporarily deployed to test January 28, 2026 18:36 — with GitHub Actions Inactive

Copilot started reviewing on behalf of chewi January 28, 2026 18:36 View session

chewi added 4 commits January 28, 2026 18:40

refactor: simplify the components.json jq code

e9c7734

jq is very powerful, you know. :) Signed-off-by: James Le Cuirot <[email protected]>

style: make "isOS" shell helpers less cumbersome to use and add isUbuntu

a4a12a2

We almost always pass `${OS}` so make this the default.

chore: update test data

ae35d90

Signed-off-by: James Le Cuirot <[email protected]>

chewi force-pushed the chewi/simplify-components branch from 09b286b to ae35d90 Compare January 28, 2026 18:40

chewi temporarily deployed to test January 28, 2026 18:40 — with GitHub Actions Inactive

Copilot AI reviewed Jan 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: simplify components.json code #7742

refactor: simplify components.json code #7742

Uh oh!

chewi commented Jan 27, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Copilot AI Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

refactor: simplify components.json code #7742

Are you sure you want to change the base?

refactor: simplify components.json code #7742

Uh oh!

Conversation

chewi commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chewi commented Jan 27, 2026 •

edited

Loading